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The EVEefainorphosis of 
Mathemat ics Education 




A major international study of 
student performance, published last 
February, used the most advanced 
statistical theories and the latest 
assessment methods to prove once 
again what educators have known for 
decades — American students do 
poorly in mathematics. 

The second International Assess- 
ment of Educational Progress (IAEP) 
reported that American 9-year-olds 
scored below students in 11 of 14 

o nations. American 1 3-year-olds in the 
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study, who were ranked against 
students in 20 nations, outscored 
only those in Brazil, Jordan, 
Mozambique, and Portugal. 

The hard-edged reality of these 
statistics is especially disheartening 
when contrasted with the idealistic 
goals of President Bush's America 
2000 education plan, which calls for 
the United States to lead the world 
in mathematics by the next century. 

Yet the assessment, conducted by 
Educationai Testing Service, 
revealed no real surprises. It merely 
confirmed the results of numerous 
studies conducted during the past 
40 years. These studies all show 
that the nation's math performance 
is deteriorating — a trend that may 
thwart America's efforts to achieve a 
competitive edge in the interna- 
tional marketplace. 

One of the most recent reports 
calling for mathematics education 
reform, is Everybody Counts, pub- 
lished in 1989 by the National 
Research Council. Due in part to the 
state of mathematics education in 
today's schools, the report says, 
"Three of every four Americans stop 
studying mathematics before com- 
pleting career or job prerequisites. 
Most students leave school without 
sufficient preparation in mathemat- 
ic5j to cope with either on-the-job 
demands for problem-solving or col- 
lege expectations for mathematical 
literacy. Industry, universities, and 
the armed forces are thus burdened 
by extensive and costly demands for 
remedial education." 

An earlier report was The Under- 
achieving Curriculum, Assessing 
School Mathematics from an Inter- 
national Perspective, published in 
1987. The report conta led the 
cheerless results of the Second 
International Mathematics Study 
and helped fuel the fire of concern 
over American mathematics 
achievement. 

Among its observations: 

• Declines in arithmetic achieve- 
ment from 1964 levels raise "ques- 
tions about the impact of the 'Back 
to Basics Movement' of the 1970s" 
and confirm the nation's "weak 
position." 

• Twenty-five million children 
study mathematics in the nation's 
schools, but their time is "largely 
devoted to mastery of the computa- 
tional skills which would have been 
needed by a shopkeeper in the year 
1940 — skills needed by virtually 
no one today...." 

Researchers have reported simi- 
lar findings in A Nation At Risk, 
a landmark publication by the 
National Commission on Excellence 



in Education, published in 1983, 
and An Agenda for Action, which 
recommended a 10-year plan for 
revising school mathematics and 
was published by the National 
Council of Teachers of Mathematics 
(NCTM) in 1980. 

Other significant reports were 
published in 1975, in 1963, and in 
1959, just two years after Sputnik I 
rattled the American academic 
community. 

But even Sputnik cannot be cred- 
ited with initiating concerns about 
mathematics education. In the first 
half of the century, experts were 
already lamenting students' poor 
performance in mathematics. 

The causes behind this continuing 
problem are numerous and varied, 
including: 

• underprepared teachers and 
students 

• outmoded textbooks and curricula 

• uninspired teaching methods 

• inadequate administrative 
support 

• uninterested parents 

• complex social problems 
Often mixed into the distasteful 

brew are either bureaucratic inertia 
— the tendency for schools to reject 
anything that seems too different, 
faddish, untested, or risky — Gr 
overzealousness — a tendency to 
plunge headlong into education 
experiments without adequate plan- 
ning or staff training. 

picing the Mix 

Many mathematics educators, 
however, are now counting on a dra- 
matic new ingredient to spice up the 
mix. The new addition is a set of 
revised standards for elementary 
and secondary mathematics curric- 
ula and teaching, developed by the 
Commission on Standards for 
School Mathematics of the NCTM. 

The new standards: 

• are based on the most current 
research on educational and work 
force needs 

• are realistic and applicable to stu- 
dents of all ages, nationwide 

• were endorsed by 15 math associa 
tions, societies, conference boards, 
councils, institutes, etc. and sup- 
ported by 25 professional organiza- 
tions concerned with teaching, 
learning, and children. 

• have the potential, if effectively 
employed, to level the playing field 
for minorities and women, who per- 
form poorly in traditional mathe- 
matics coursework. 
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In 1986, the NCTM published 
Curriculum and Evaluation 
Standards for School Mathematics, 
which sets forth recommendations 
for revisions in what children 
should know and be able to do in 
mathematics from kindergarten 
through high school. The group 
has also developed innovative yet 
practical teaching methods and 
suggestions for teacher training, 
described in their publication, 
Professional Standards for Teaching 
Mathematics, 

The introduction to Curriculum 
and Evaluation Standards states 
the rationale for the new standards: 

"Schools, as now organized, are a 
product of the industrial age. In 
most democratic countries, common 
schools were created to provide most 
youth the training needed to become 
workers in fields, factories, and 
shops.... The educational system of 



the industrial age does not meet the 
economic needs of today. New social 
goals for education include (1 ) math- 
ematically literate workers, (2) life- 
long learning, (3) opportunity for all, 
and (4) an informed electorate. 
Implicit in these goals is a school 
system organized to serve as an 
important resource for all citizens 
throughout their lives, , . . As society 
changes, so must its schools," 

The new goals ror students in 
mathematics, as explained in the 
Standards, are: "(1) that they learn 
to value mathematics, (2) that 
they become confident in their 
ability to do mathematics, (3) that 
they become mathematical problem 
solvers, (4) that they learn to 
communicate mathematically, and 
(5) that they learn to reason 
mathematically." 

The new standards also empha- 
size that in order to make mathe- 
matics education useful and accessi- 
ble to more students, the central 
focus should be on problem solving. 
"Problem solving is not a distinct 
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topic but a process that should per- 
meate the entire program and pro- 
vide the context in which concepts 
and skills can be learned." 

lb do all of this, the standards call 
for classes that: 

• are creative 

• emphasize comprehension 
and problem solving, not just 
memorization 

• train students to use calculators 
or computers effectively to enhance, 
not replace, knowledge of basic skills 

• use manipulative materials to 
promote maximum comprehension 

ew Standards in Action 

Already, many teachers have 
adopted these concepts. When they 
do, their classes often begin to look 
like the fifth grade advanced math 
class taught by George Tiefenthaler 



of the New Hope-Solebury Elemen- 
tary School in Pennsylvania. 

A morning spent in class with 
Mr. T., as his students call him, is 
like previewing the future — the 
class embodies the qualities that 
education leaders are seeking for all 
mathematics classes. 

Mr. T. grabs the students' atten- 
tion at the very start by standing on 
a chair at the front of the classroom, 
waving his basketball-player-sized 
hands like semaphores. This is one 
of Mr. T.'s games, and the children 
know that his hand positions signal 
that the students should add, sub- 
tract, multiply, or divide the number 
of fingers he is showing. With this 
deceptively simple strategy, Mr. T. 
transforms the drudgery of mental 
exercises into an appealing but chal- 
lenging competition that requires 
concentration and a thorough 
knowledge of arithmetic skills. 

After this warm-up review, Mr. T. 
gives the students a few problems 




to work on their school-supplied 
calculators. 

He then whisks the class into an 
exercise called "the problem of the 
day," instructing the students to 
work the problem alone first and 
then discuss it with classmates. 

The problem states: "Your friend 
asks you for change for a dollar. You 
tell your friend that although you 
have $1.15 in change, you cannot 
make change for a dollar. Your 
friend says that you certainly can. 
Who is right? Explain." 

Even before Mr. T. explains the 
ground rules, one bright student 
raises his hand and states, "But 
both people could be right." 

"Right," says Mr. T., a smile cross- 
ing his face. The comment confirms 
that the children are learning the 
most important lesson of the year — 
there are many different ways to 
find an answer to a problem and, 
often, many different answers 
as well. 

UP, DP, Do It!, DIMS 

Before Mr. T. sets the class off in 
search of an answer, he reviews his 
recommended problem-solving 
steps. The words and symbols for 
the steps are written on paper-and- 
string mobiles, constructed by the 
students early in the year, now sus- 
pended from the ceiling and flutter- 
ing overhead. 

The first step, symbolized by an 
arrow and the letters UP, is Under- 
stand the Problem. The second step 
is DP — Devise a Plan. The next 
step is represented on the mobiles 
by a Nike sneaker, symbolizing the 
Nike television commercials that 
feature the slogan, "Just do it!" 
For the students, this means to 
work the problem. Finally, the paper 
mobiles proclaim DIMS, which 
stands for, Does It Make Sense? 

"Look at your answer and see if 
it makes sense," says Mr. T. "Is it 
possible for the boy to have change 
for a dollar?" "Yes," answers the 
class. He could have four quarters, 
one dime, and one nickel, or sev- 
eral other combinations of coins. 

"Is it possible for him to not 
have change for a dollar?" "Yes." 
The class then comes up with sev- 
eral ways to have $1.15 in change 
and yet not have exact change for 
one dollar. The boy might, for 
instance, have nine dimes and one 
quarter, or three quarters and four 
dimes, or one half dollar, one quar- 
ter, and four dimes. 



Mr. T. caps off his class with one 
of the key components of the new 
mathematics standards — manipu- 
latives. For the day's lesson, he 
passes out handfuls of black disks 
that look like flat, smooth checkers 
and creates groups of four to five 
students to work together to solve a 
problem. "No, you can't play poker 
with them," he says laughing, in 
answer to one student's comment. 

He then hands out diagrams 
showing dots arranged in rows of 
varying lengths to form a diamond 
shape and instructs the students to 
arrange their disks in the pattern 
shown on the diagram. "The prob- 
lem is," he says, "how many ways 
can you find to count them?" After 
the first few minutes he even gives 
the students the easiest method — 
count the dots one by one. 

The disks are like a magnet for 
the children's hands. They cannot 
resist touching and moving them, 
arranging them into the pattern on 
the hand-out, then rearranging 
them again and again. 

One boy calls out, "Why bother? 
There are only three ways we can 
do it." But Mr. T. merely says, 
"We'll see." 

The noise level in the classroom 
rises, but Mr. T. does not call for 
quiet. The children are exploring, 
sharing, and comparing ideas, tell- 
ing each other how to move the 
disks into new patterns. 

After a while, Mr. T. asks the 
groups to share their discoveries 
with him, and he writes them all on 
the blackboard. 

• First method: Change the 
arrangement from a diamond into a 
square. Then count how many rows 
have four disks (four of them) ana 
how many rows have three disks 
(three), multiply out, and add 

the totals, 

• Second method: Count the center 
row of the diamond (seven disks), 
end then count all the disks to the 
riftht of the center (nine). Because 
the pattern is symmetrical, there 
are as many disks to the left as to 
the right of the center line, so you 
can multiply nine by two and then 
add the seven of the center line. 

• Third method: Group them in sets 
of five, and multiply by the number 
of groups (five times five). 

The class comes up with several 
other methods as well, excited by 
the number of possibilities. 

"What is the point? What is the 
teacher trying to tell us?" Mr. T. rhe- 
torically asks the class. "There is 
more than one answer to the prob- 
lem," he tells them. "Your homework 
assignment for tonight is to design a 
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problem using 36 marbles, and to 
find as many ways as you can to 
count when you have 36." 

Class is over for the day, and 
the children rocket off to the next 
subject. 

ew Insights into Learning 

Mr. TVs emphasis on solving prob- 
lems through creative thinking dis- 
tinguishes him from the more tradi- 
tional mathematics teachers who 
emphasize rote memorization of 
arithmetic facts. While Mr. T. also 
requires his students to memorize 
basic facts, he adds additional activ- 
ities that help them understand 
underlying concepts. The students, 
in other words, learn to reason 
mathematically — a skill they can 
use throughout their lives. 

Mr. T.'s emphasis on reasoning 
and problem solving are essential 
elements of the new standards for 
mathematics. 

Many elements of the new stan- 
dards were developed as a result of 
new insights into learning itself, 
says Robert Davis, a noted mathe- 
matics educator and researcher at 
Rutgers University. In his 1984 
book, Learning Mathematics, The 
Cognitive Science Approach to 
Mathematics Education, Davis 
writes that these insights were 
drawn "from Piaget's observations of 
children, from computer scientists' 
development of 'intelligent' com- 
puter programs, and from research 
into human information-processing 
capabilities." 

Twenty years ago, there was no 
extensive body of research that told 
you, for instance, how kids really 
think about fractions," confirms 
Richard Lesh, a research scientist at 
Educational Testing Service (ETS). 
In the absence of that research, 
"people with decent intuitions about 
such things tried to guess, but they 
missed the boat some of the time." 

Since then, however, "people went 
out and watched kids solving real 
problems, watched them working 
with concrete materials," Lesh 
says. As a result, they found that 
some of the tenets of behavioral psy- 
chology, which were the foundation 
for much contemporary educational 
practice, were incorrect. Their 
research pointed them toward new 
understandings of human learning, 
now called cognitive psychology. 

The difference, Lesh explains, is 
that "behavioral psychology was 
based on rules — everything got 
reduced to a rule of behavior, an 



action even, for the most part. 
Learning meant learning rules. 

"Cognitive psychology is about the 
fact that people interpret the world 
using internal models, metaphors, 
and stories," Lesh says. People use 
these stories to help them under- 
stand and place in context their own 
experiences. 

"Research in learning," states 
the report Everybody Counts, 
"shows that students actually con- 
struct their own understanding 
based on new experiences that 
enlarge the intellectual framework 
in which ideas can be created. Con- 
sequently, each individual's knowl- 
edge of mathematics is uniquely 
personal. Mathematics becomes 
useful to a student only when it 
has been developed through a per- 
sonal intellectual engagement that 
creates new understanding. Much 
of the failure in school mathemat- 
ics is due to a tradition of teaching 
that is inappropriate to the way 
most students learn." 

"It's not that kids don't know 
anything and then we teach them 
something," Lesh explains. "They 
have a lot of ideas, and what we 
need to do if we want to teach 
them is to first make contact with 
the way they think, and then mold 
and shape those ways in directions 
we want to go." 

aking Connections 

The recognition that mathematics 
is learned most effectively when it is 
placed in a personal context makes 
the new standards significantly 
distinct from current practice in 
mathematics education. 

When students are able to link 
theory with reality, Lesh explains, 
they are better able to understand 
that mathematics is not just 
"a bunch of rules that people use for 
doing calculations with numbers. 
Mathematics is really about models 
for thinking about the world — 
models that deal with quantities, 
shapes, relationships, and other 
mathematical things." 

At its best, mathematics educa- 
tion helps people understand these 
models so they can apply them in 
different situations. 

In a course he conducts at 
Rutgers University, Lesh leads his 
students, mainly teachers, through 
a series of exercises to help them 
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better understand the foundations 
of mathematics. 

"At the end of the course, the 
teachers end up saying something 
like, 'Math is really about models, 
and my job as a teacher is to help 
my kids construct those powerful 
models/ That's enormously differ- 
ent than, 'Math is a bunch of rules 
and I want my kids to do them 
flawlessly.' " 

Davis agrees. During an interview 
at his office at Rutgers, he points 
out that mathematics "shouldn't be 
thought of as a few very specific 
things that you memorize and you 
do without understanding them — 
it's much more like an art or 
a craft." 

As an example of the problem- 
solving nature of mathematics, 
Davis cites a teacher who developed 
an exercise that asked school chil- 
dren to gather several leaves of 



varying sizes and shapes. She would 
then ask the children to develop a 
method to compare the sizes of 
the leaves. 

There is no formula for figuring 
out such a problem, Davis explains. 
Students must devise their own 
models and methods to find an 
answer. The goal of the new stan- 
dards for mathematics is to train 
students for creative problem solv- 
ing of this sort, so they can do more 
than just calculate number prob- 
lems on a ditto sheet. 

The need for reality-based mathe- 
matics is not a new concept, 
Davis says. "It's been developing for 
a long time," 

"The fact that people have been 
learning these abstract procedures 
in a meaningless way," Davis says, 
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"is not new knowledge — we've 
known it for ages. And the fact that 
you ought to deal with math in a 
meaningful context is truly not a 
new idea. It certainly goes as far 
back as John Dewey, and I don't 
think he created it, either." 

The system of using rote memori- 
zation developed in the days when 
children generally were brought up 
to follow their parent's profession, 
Davis theorizes. "In those days, doc- 
tors were mostly the sons of doctors 
and farmers were typically the sons 
of farmers. If you grew up on a farm, 
you knew what farmers do, and 
things didn't really change that fast 
for a long time." 

A farmer's son knew that to put 
up a fence around a field, he would 
have to know a formula to deter- 
mine the perimeter, Davis explains. 
He would use another formula to 
determine the area of the field, so he 
could estimate how much seed he 
needed. The numbers themselves 
stood for concepts familiar to stu- 
dents from their daily lives, making 
the ideas both easier to understand 
and to learn. 

"Kids in school nowadays," Davis 
3ays, "do not know the context, and 
they probably are not going to follow 
a parent's occupation, or they find 
out the occupation is different by 
the time they get into it. They can't 
count on the context being there." 



Yet long after the context was 
removed, the education system con- 
tinued using the old methods 
because they had worked so well in 
years past. "But without the con- 
text, the numbers by themselves are 
meaningless. You don't under- 
stand what you are actually doing," 
Davis says. 

The context argument is purely 
conjecture, Davis emphasizes, but 
it makes sense in light of recent 
research. Davis cites the example of 
a researcher who wouid give people 
problems in a class situation and 
later present a similar problem in a 
situation such as grocery shopping 
or adjusting recipes. The researcher 
found that even people who could 
not work the problems in a class- 
room could often figure them out 
when they were presented in a real- 
life context. 

"There's no doubt, I think, that 
these contexts are very powerful," 
Davis emphasizes. 

"If you start looking at what ycu 
do with math, you start seeing that 
you typically use it in a context. 
People don't come up to you in the 
ordinary world, and say, 'What's this 
divided by that,'" Davis says. 
"They will ask a question such as, 
•We've got this many kids going on a 
school trip — how many school 
buses do we need?' 

"So really, what we're arguing for 
is that math should be in context, 
that it should involve manipulable 
materials, that people should learn 
to think through problems that no 
one showed them how to do. Cer- 
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tainly no one showed me anything 
about refinancing a mortgage," 
Davis says. 

gjjj| laying Tug-of-War with Rote 

Despite the collaborative process 
that the NCTM used in developing 
its new standards for mathematics 
education, some educators still 
object to its goals and methods. 

This dispute is part of what 
Davis, in his book, Learning Mathe- 
matics, calls a continuing 
tug-of-war in mathematics educa- 
tion "between a *drill-and-practice' 
and 'back-to-basics' orientation that 
focuses primarily on memorizing 
mathematics as meaningless rote 
algorithms, versus an approach 
based on 'understanding* and 'mak- 
ing creative use' of mathematics." 

Although current research sup- 
ports the creative approach to edu- 
cation, in practice the tug-of-war is 
still being won by the traditional 
model despite the fact that u it is 
quite clear that present pract 
at an ineffective extreme," states 
the report Ei rrybody Counts. 

Advocates of the new standards, 
of course, believe that the balance of 
power in the tug-of-war will gradu- 
ally change as both schools and col- 
leges shift away from "classrooms of 
passive students who are expected 
to sit and absorb rules which appear 
as arbitrary dicta from on high. . . ." 

In their place, they hope to see 
classes that emphasize "broad-based 
mathematical power," says Every- 
body Counts. 

"Mathematical power requires 
that students be able to discern 
relations, reason logically, and use a 
broad spectrum of mathematical 
methods to solve a wide variety of 
non-routine problems. The reper- 
toire of skills which now undergird 
mathematical power includes not 
only some traditional paper-and- 
pencil skills, but also many broader 
and more powerful capabilities." 

Some educators are still pulling 
the tug-of-war rope for rote memori- 
zation, but in many cases, their 
objections arise from a failure to 
fully understand the standards. 

One retired math teacher 
described his views in a letter to the 
editor of the NJEA (New Jersey 
Education Association) Review last 
fall (November 1991). 



"You have to crawl/walk before 
you can run" the letter states. "If 
formulas aren't memorized, there . 
will be no basis for the mathemati- 
cal reasoning. If there is no mecha- 
nistic answer finding, there will be 
no conjecturing, inventing, and 
problem solving. If you don't know a 
body of so-called isolated concepts 
and procedures, there won't be any 
connecting mathematics and its 
applications. Judicious use of old- 
fashioned rote memory and drill are 
as necessary today as they were in 
generations past." 

The standards, however, recog- 
nize the need for "a certain amount 
of basic memorization," explains 
John Dossey, professor of mathe- 
matics at Illinois State University 
and former president of the NCTM. 

"We expect kids to know basic 
facts and operations as much today 
as we have at any time in our his- 
tory. What we're saying is that what 
children do learn, they need to con- 
nect with the real world, too." 

In general, objections to the new 
standards are unusual, Dossey says. 
"You'll find a few people, but 1 think 
there is general consensus because 
of the way the standards were 
developed. There just is no really 
identified large group of resistance 
out there. 

"Personally, although I was very 
involved in developing the stan- 
dards, there are places I can point to 
and say, T don't agree with that,' 
but, overall, I wouldn't have the 
slightest hesitation for my child to 
complete a program using the new 
standards." 

gjj ifferent from "New Math" 

For all of their potential, today's 
new mathematics reforms are still 
often misunderstood by the public 
and by some educators who may be 
less involved in such professional 
organizations as the NCTM. In part, 
perhaps, this is because they 
remember another "new math," 
associated with ambitious mathe- 
matics education reform programs 
of the late 1950s through the 1970s. 

There are, however, a "whale of a 
lot of differences," as Dossey says, 




between today's mathematics 
reforms and the so-called "new 
math." 

"The 'new math' probably tried to 
throw out too much and to start 
from scratch, including throwing 
out the established ways of dealing 
with basic operations," explains 
Nancy Cole, vice president of Educa- 
tional Testing Service and a member 
of the Mathematical Sciences Edu- 
cation Board. "That's how it got the 
name 'new math"* 

In contrast, the current reform 
movement builds on existing 
approaches and knowledge and 
expands out from them. 

Today's mathematics also takes 
into account the fact that "children 
come to school with a certain 
amount of mathematical back- 
ground, and you're better off trying 
to build their mathematical under- 
standing on that background.' "as 
opposed to just throwing rules and 
procedures at them, out of context." 
says Beverly Whittington. an ETS 
senior examiner in the College 
Board program. 



"That doesn't mean the children 
are left on their own to understand 
math." Dossey explains, "The 
teacher and the curriculum provide 
them with experiences that help 
them shape their understanding in 
a progressive fashion." 

The curriculum, he says, helps 
them "work to create their own 
understanding. They learn not just 
because the teacher tells them its 
so. but because they see it and wres- 
tle with it on their own." 

The goals of today's math make it 
perfectly suited to this type of expe- 
riential teaching, agrees Whitting- 
ton. "The old 'new math' was more 
theoretical. The idea then was to 
give people an idea of the structure 
of mathematics — it was more pure 
mathematics. 

"Today, people are talking about 
putting things in a real-life context 
and being able to relate mathemat- 
ics to other fields. I think that s to 
help the children transfer their 
mathematics concepts more easily. 
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so they can see the worth of mathe- 
matics in their lives." 

Another feature of "new math," 
Dossey points out, is that it aspired 
to ''create mental masters over- 
night." Because the reforms were 
spurred by an urgent need to catch 
up with the Soviet Union's scientific 
advances, they tried to make "over- 
night" changes, he says, "like taking 
a template and placing it down on 
the grades and tracing around it." 
The system basically told the 
students, "Here's the structure you 
need, now learn it" with the goal of 
"pumping the stuff in quick." 

Today's mathematics reforms, in 
contrast, are being introduced into 
curricula slowly. "If it happens too 
quickly then there's a problem." 
Dossey says. 

"Here, we're talking about a 
staged change, over a decade, per- 
haps, so that teachers are comfort- 
able with it. The children are not 
expected to change overnight, but to 
change as they move through the 
svstem." 



Perhaps the most basic difference 
between the two movements, Davis 
points out, is that while today's 
reforms were developed with a uni- 
fied structure, the "new math" was 
not. He contends, in fact, that while 
mathematical theory did, indeed, 
support many of the mathematics 
reforms from the '50s through the 
'70s, the term "new math" is errone- 
ous — promulgated not by educa- 
tors, but by the media. 

The single common element 
among these reforms, Davis says, 
agreeing with Cole, was that "every 
one of them wanted to replace the 
existing math book with something 
else.' 1 Otherwise, he says, the reform 
efforts were quite diverse. 

"If one were to believe popular 
accounts, one might conclude that 
something known as 'the new math- 
ematics' was created in the late 
1950s and early 1960s, that it was 
tried in U.S. schools, and that it 
failed disastrously." Davis writes in 
Lea rn intf Ma them at ics . 

"This is all quite wrong. 

"In the 3 950s and 1960s, manv 
different school curricula were 
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created. . . . These programs were not 
all similar — some were mainly 
abstract, some used mainly concrete 
learning experiences; some stayed 
close to traditional content, some 
explored very different content; 
some dealt only with mathematics, 
some combined mathematics and 
science, some approached mathe- 
matics mainly through science; 
some assumed considerable rote 
learning, others assumed very little; 
some emphasized 'discovery 
learning,' others used it little or not 
at all; some undertook extensive 
teacher education programs, some 
took an opposite route and sought to 
prepare so-called 'teacher-proof' 
material." 

Clearly, this was no unified set 
of standards that could be easily 
categorized as "new math." 

In fact, Davis writes, "Most 
schools experienced little or no 
change. " 

Not only was there no single "new 
math," there also was no complete 
and disastrous failure, Davis writes. 
"In those few schools wher^ they 
were extensively and carefully 
implemented, the best of the new 
curricula produced very pronounced 
gains in student performance. The 
fact is that they did not fail; on the 
contrary, they were markedly 
successful. ..." 

With the combination of some 
significant successes and some 
major problems, the overall results 
of the curricular reforms character- 
ized as "new math" were "quite 
uneven," concludes the report 
Everybody Counts. 

What Made it Happen? 

"A major difference between the 
new math and the reforms being 
suggested now is that this time 
around, educators worked carefully 
on building a broad consensus," 
explains ETS's Cole. 

"I think there must have been a 
critical mass of really concerned, 
respected, and wise mathematics 
educators. It wasn't a single person, 
but it was a strong cluster of them 
that saw the possibilities for chang- 
ing mathematics education and 
changing the political processes 
necessary to have an impact. 

"One of their early steps was to 
bring many professional groups 



together to discuss curriculum, 
which gave them a lot more power. 
Then they went to the National 
Academy of Sciences and in 1985, 
got the academy to start the Mathe- 
matical Sciences Education Board 
(MSEB) to be a coordinating board 
for all the professional groups " 

The National Council of Teachers 
of Mathematics, which took the lead 
in creating the standards, had 
strong teacher and college-level 
mathematics educator leadership, 
Cole says. At the same time, the 
MSEB worked with the groups to 
have the standards endorsed. 

"It was just a lot of really strongly 
committed people willing to work 
very hard that made it happen," 
Cole says. 

Adding impetus to the movement 
for change, Dossey says, were two 
reports on the nation's failing math- 
ematics achievement levels: 

• A Nation At Risk, published in 
1983 by the National Commission 
on Excellence in Education, "really 
shook up the education community," 
Dossey says. 

• The Underachieving Curriculum, 
Assessing School Mathematics from 
an International Perspective, pub- 
lished in 1987, showed that neither 
the "new math" reforms nor the 
"back-to-basics" movement that fol- 
lowed had improved American 
mathematics achievement levels. 

gjj preading the Message 

Despite its drawbacks, the "new 
math" taught educators some valu- 
able lessons. Among the most impor- 
tant, states Everybody Counts, is 
that ". . . any successful effort to 
improve mathematics curricula and 
instruction in the schools will 
require an extensive public informa- 
tion campaign that reaches all the 
varied constitutencies of mathemat- 
ics education." 

The lack of communication was a 
major roadblock for the "new math" 
reforms. As Davis explains, "there 
was never any intelligent discussion 
of it." Very little was written about 
"new math" when it was first intro- 
duced, so few educators clearly 
understood the rationale behind the 
programs and few participated in 
discussions with their peers or with 
the public about how best to accom- 
plish their goals. 

Today's reforms have an advan- 
tage in this area because many pro- 
fessional organizations were 
involved in their development. The 
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"Unless texts. . . change dramatically to 
reflect the standards 5 philosophical 
underpinnings. . . there will be little, if any, 
change in the way that math is taught. . . ." 



NCTM mailed an executive sum- 
mary of its Standards reports to 
every principal and school board 
president in the nation, Dossey 
says. 

But even with national support, 
Cole points out, dissemination prob- 
lems remain. "Not every teacher 
belongs to NCTM, or is active, so 
youVe got just a huge gap between 
these national groups who are doing 
very constructive things and very 
busy teachers in classrooms who 
may not even know that some of this 
is going on." 

lb reach a wider audience, Cole is 
a member of a state coalition exam- 
ining various education activities 
for teachers and parents, ranging 
from public TV programs to pam- 
phlets for teachers about using the 
standards in the classroom. Called 
the New Jersey State Mathematics 
Coalition, the group will collect and 
disseminate information about suc- 
cessful programs in use nationwide. 
The committee is also opening con- 
tact with school administrators "to 
get a feel for how we can help them 
and how they can help us." 

"Administrator groups are con- 
cerned about reform, just like 
teacher groups are at this stage, so 
the climate's right for mutual help," 
Cole says. "Up to this point, weVe 
done very little with principals, 
superintendents, and administra- 
tors, but we're beginning to realize 
that they're very important groups 
to reach." 

Committees similar to the New 
Jersey Coalition are being formed in 
all 50 states at the initiative of the 
MSEB. 

The advocates of change will still 
be fighting an uphill battle, how- 
ever, until the textbooks are in 
agreement, Dossey says. "There are 
already some textbooks that are 



moving in that direction, but text- 
books will only change when the 
market demands." 

To discuss the need for revised 
texts, the NCTM met with the 
School Division of the Association of 
American Publishers last March. 
During the meeting, NCTM mem- 
bers told the textbook publishers 
that, "Unless texts and other mate- 
rials change dramatically to reflect 
the standards' philosophical under- 
pinnings.. .there will be little, if any, 
change in the way that math is 
taught," said an article in Education 
Week. 

The article quotes Glenda Lap- 
pan, a mathematics professor at 
Michigan State University, as tell- 
ing other university researchers, 
"If we don't put our focus on writing 
better materials for teachers, we 
will have missed the boat. We will 
not serve the needs of teachers until 
we write better materials from 
which teachers can learn." 

ccepting Calculators 
and Computers 

One element of the new mathe- 
matics reforms that has received 
considerable attention in both schol- 
arly and popular literature is the 
recommendation that new technolo- 
gies be included in mathematics 
curricula. 

Because these technologies are 
now integral to mathematics, the 
NCTM charged the committee 
developing the new standards to, 
"Create a coherent vision of what it 
means to be mathematically literate 
both in a world that relies on calcu- 
lators and computers to carry out 
mathematical procedures and in a 
world where mathematics is rapidly 
growing and is extensively being 
applied in diverse fields." 

Yet the use of calculators has long 
been debated. Advocates say that 




16 



calculator use will pare the time 
spent on routine computations and 
help students concentrate on mathe- 
matical concepts and applications. 
Critics contend that calculators will 
allow students to avoid learning the 
basics of mathematics they will 
need beyond their school years. 

"The mathematics professional 
community now is quite well unified 
that calculators and computers will, 
in the future, be like paper and pen- 
cil," says Cole. "We need to teach 
kids when to use them, when not to 
use them, and how to use them 
effectively. It is essential that those 
kinds of tools be part of mathemat- 
ics education." The public often mis- 
understands how educators plan to 
incorporate calculators in the class- 
room, Cole says 

As the report Everybody Counts 
states, "Calculators and computers 
for users of mathematics, like word 
processors for writers, are tools that 
simplify, but do not accomplish, the 
work at hand. Thus, our vision of 
school mathematics is based on the 
fundamental mathematics students 
will need, not just on the technologi- 
cal training that will facilitate the 
use of that mathematics. " 

Contrary to the fears of many, the 
report says, there is "no evidence" 
that the availability of calculators 
and computers "makes students 
dependent on them for simple 
calculations." 

"As long as the primary focus is 
their use as a tool, they should pose 
no threat to mathematics educa- 
tion," says Chancey Jones, principal 
measurement specialist at ETS. 

In fact, he notes, today's calcula- 
tors have already progressed far 
beyond simple arithmetic functions. 
"They can be a teaching instrument 
because the students can see on the 
screen what it means to approach a 
limit or draw the curve of an equa- 
tion. It's right in front of them; they 
can interact with the material. For 
all practical purposes, a lot of the 
graphing calculators today are 
really minicomputers." 

The public's fears are based on a 
general misunderstanding of mathe- 
matics itself, says Cole. "They think 
of mathematics as learning to add, 
but that's not mathematics — that 
kind of operation is a very small and 
insignificant piece of mathematics. 
One of the big problems we fight is 
that people have this notion that 
mathematics is mainly those 
operations." 



Calculators are not just for work- 
ing computations in class; they 
affect society at large, points out 
Lesh. "The calculator and computer 
relieve you of doing the tedious com- 
putation, but those tools also intro- 
duce whole new things that you 
need to understand, such as graph- 
ics-based communication." 

Because graphics are now so easy 
to produce on calculators and com- 
puters, their use has increased enor- 
mously in fields such as business. 
Now routine features in popular 
newspapers and magazines, graph- 
ics are used to illustrate a variety of 
issues such as trends or cost-effec- 
tiveness. Graphics have vastly 
improved the way people can com- 
municate mathematical ideas, 
Lesh says, and they therefore need 
to be included in mathematics 
classrooms. 

|7| evised Assessments 

The new standards for mathemat- 
ics call for changes in the way math- 
ematics is taught as well as the con- 
tent that is emphasized. Although a 
significant amount of traditional 
material remains in the recom- 
mended curriculum, the standards 
require new assessment strategies, 
points out James Braswell, princi- 
pal measurement specialist at ETS. 
Braswell heads the mathematics 
areas of the Preliminary Scholastic 
Aptitude Test (PSAT) and the Scho- 
lastic Aptitude Test (SAT). 

"The standards have deempha- 
sized some of the more mechanical 
sorts of things that students have 
spent a lot of time on in the past," he 
says. "They emphasize other things 
— for example, working with data, 
making graphical representations of 
data, and making connections 
between different disciplines, such 
as geometry and algebra." 

All this means major, if gradually 
introduced, revisions in standard- 
ized assessments, Braswell says. 
"We have made it a point to get all of 
our test developers familiar with the 
standards so they know what is 
being emphasized and what is being 
deemphasized, but change will 
have to be gradual, giv our oper- 
ating parameters." 

Available testing time, validity, 
and reliability are among the 
parameters that must be consid- 
ered. "With the SAT, we have to 
come up with a reliable measure of 
what students can do in an hour. 
There are two possible approaches: 
you can pose three or four fairly 
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complex problems, so you have all of 
your eggs in just a few baskets, or 
you can come up with a lot of little 
problems and hope that based on 
the responses, you can make some 
inference about the larger picture. 
That's the way we're doing it now, 
and I think that's probably the way 
it will be for a while, but advances 
in computer-delivered testing and 
expert systems hold promise for 
the future.* 

The large standardized tests are 
also obliged to reflect current teach- 
ing practices and curricula, points 
out ETS's Jones. "The standards are 
being widely distributed, but it's 
going to be a long time until we 
know whether or not they have a 
real impact in the classroom." 

A more complex problem confront- 
ing test developers is validity, 
Braswell says. "One of the objectives 
of the standards is providing stu- 
dents with the opportunity to solve 
more realistic problems, problems 
that go beyond a quick answer, and 
problems that people can work on in 
groups. The standards point out 
that if you go out in business and 



industry, you find people working in 
teams and collaborating. 

"But that's hard to do on a test," 
Braswell points out. "When someone 
is taking the SAT, we frown on a col- 
laborative effort taking place — how 
would you know who did the work? 
Yet the new standards emphasize 
how people learn from a collabora- 
tive process, not necessarily who 
gets credit for what lovei of involve- 
ment. In a classroom, if two people 
work together, even if one person 
does most of the work, the other per- 
son has learned something as a 
result of that collaborative effort. * 

So far, Braswell says, very little 
new material has been developed 
that meets all the objectives of the 
standards in terms of tasks the stu- 
dents have to do, texts that include 
and emphasize these tasks, or 
model test questions that reflect 
the standards. 

In an effort to accommodate the 
new standards, ETS is already 
redesigning its largest testing pro- 
gram, the SAT. The new tests will be 
introduced in the spring of 1994. 

Among the changes: 
• Some new questions will require 
students to produce their own 
responses, instead of choosing the 
standard multiple-choice answers. 
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• Calculators will be permitted 
(although not required). 

• The assessment will place 
increased emphasis on interpre- 
tation of data and applied mathe- 
matics, 

"What we ought to strive for in 
the testing programs is a real 
balance of both well-structured 
open-ended questions and multiple- 
choice questions that allow students 
to demonstrate their abilities and 
capabilities," says Jones. "I don't 
want people to think that there's no 
value at all to multiple-choice ques- 
tions. They have served us well. 
And there are complications of 
validity and reliability when you 
use only open-ended questions." 

A good example of an assessment 
that is balanced in terms of a 
variety of item formats and is 
responsive to the changing direc- 
tions in mathematics education, 
Jones says, is the 1992 National 
Assessment of Educational Progress 
(NAEP) Mathematics Assessment. 
Conducted by ETS, the assessment 
was administered to students in 
grades 4, 8, and 12. 

More than one-third of the 
questions on the assessment 
required the students to construct 
their own reponse^*. which took 
about 40 percent ot their total time. 
Some of these questions required 
the students to work with manipu- 
latives or to use rulers, protractors, 
or calculators. Still other questions 



asked students to provide reasons to 
justify their solutions. 

The assessment was unique 
because it included, for the first 
time, several extended constructed- 
response tasks at each grade level 
and allowed students to demon- 
strate their level of mathematics 
understanding within a specific con- 
text, Jones says. 

This item type enhanced the 
assessment by providing more 
detailed information about the 
students' approaches to problem 
solving, the breadth and depth of 
their mathematical understanding, 
and their ability to solve nonroutine 
problems that required higher-order 
thinking skills. 

Classroom practice, of course, 
is still the key to effecting the 
changes required by the standards. 
"The standards put a fair amount of 
the burden on the schools them- 
selves to deal with this issue, 
because they realize that it's up to 
the s r' u 's to have their own inter- 
nal, comprehensive assessment 
plan," says Brasv/ell. 

"In a classroom situation, 
students can have whatever time 
they need. They don't have to go to a 
test center under standardized 
conditions. Students can work coop- 
eratively on projects that take sev- 
eral df ,f *~ to complete. So it is likely 



that schooLs will assume the lion's 
share of the burden for the larger 
assessment projects. 

"We have to make sure that as a 
testing organization, our evaluation 
is consistent with their efforts. W * 
don't want to suggest that schools 
should be responsible for these com- 
prehensive, insightful, creative 
projects while testing organizations 
emphasize topics like factoring — 
that would be totally out of align- 
ment. But there are interesting and 
challenging problems that we can 
pose, too, that would be in line with 
the standards," Braswell says. 

"What we hope for is that if the 
schools do their job, and if we do our 
job, then test scores will go up and 
everyone — the public, employers, 
colleges — will say, The students 
know a lot more today, they don't 
just reach in and pull out these little 
canned algorithms that they've 
learned. They know how to think, 
and to solve problems, and they 
can operate in a late 1900s 
environment.'" 

fTS ooratnated Programs 

The dilemma of assessments 
also goes beyond determining who is 
the best person in the talent pool, 
Lesh says. tt We need to develop the 
talented people that we find, but we 
also need to find more of these 
talented people." 

Working for the past two years 
with a grant from the National 
Science Foundation, Lesh has 
developed a new mathematics 
assessment system that is the proto- 
type for a new ETS product called 
Packets and is designed to mesh 
with the new standards for mathe- 
matics education. 

The Packets system is a series 
of "math-rich newspapers," Lesh 
explains. The four-page tabloid- 
sized newspapers include reprinted 
articles from real newspapers that 
discuss math-related topics, such as 
grocery prices or business trends. 

Teachers can use the newspapers 
as classwork, as homework, or as 
assessments. They meet the new 
standards in that: 

• They are based in real-life 
situations. 

• They provide opportunities for 
the teachers to explore students' 
problem-solving skills. 



• They are interesting, which is 
especially helpful for students who 
do not perceive themselves as math- 
oriented. 

"We'd like the teachers to take 
this material and use it to help iden- 
tify kids with a different sort of 
mathematics talent," Lesh says. 
He explains that pilot tests have 
shown that this nontraditional 
program often helps teachers 
identify talented students who do 
not show up in traditional assess- 
ments, particularly girls and minor- 
ity students. 

Algebridge is another innovative 
assessment program that is espe- 
cially beneficial for these students. 
Developed by ETS and the College 
Board, Algebridge was designed to 
help students master the change of 
thinking required in algebra — 
a subject that is often a stumbling 
block for college-bound students. 

The Algebridge program materi- 
als "actually promote learning 
rather than just measuring 
the end results of the instructional 
process," says an article in ETS 
Developments. 

"Algebra is a critical gateway 
through which high school students 
must pass if they are to pursue 
college-preparatory course work," 
says Paul Ramsey, Algebridge 
project director. By easing the 
transition from the concrete con- 
cepts of arithmetic to the abstract 
thinking of algebra, Algebridge 
makes the path to college smoother, 
particularly for the average or at- 
risk student. 

A new program called Pacesetter 
has also been initiated by the Col- 
lege Board and ET^ o improve 
instruction by coordinating curricu- 
lum and assessment. The first Pace- 
setter offering will be in mathemat- 
ics and is scheduled for pilot testing 
in 1993-94. It will be followed by 
offerings in English, science, world 
history, and Spanish. 

The Pacesetter course represents 
the fourth year of a high school 
mathematics curriculum, as out- 
lined in the NCTM's Standards. 
Algebridge, in contrast, focuses on 
the earliest aspects of high school 
mathematics, explains Ernie Kim- 
mel, executive director of Academic 
Services in the College Board Divi- 
sion at ETS. 

The NCTM standards, Kimmel 
points out, provide general sugges- 
tions for teaching methods and 
course content, but do not provide 
teachers with specific recommenda- 
tions for curriculum or assessments. 
Teachers traditionally base their 
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coursework on the textbook they use 
and therefore "frequently choose 
to use the very superficial tests 
provided with the textbooks," 
Kiramel says. 

The Pacesetter program will pro- 
vide a better alternative, he says, 
because it was developed by a task 
force of school and college mathe- 
matics teachers chosen in consulta- 
tion with the NCTM and the Mathe- 
matical Association of America. 

Pacesetter mathematics includes 
an outline of course con nt, 
"meaty" coordinated assessments 
that require students to apply their 
knowledge, and coordinated teacher 
development opportunities. 

All the material is based on what 
professional educators in the subject 
area feel that students should know 
and be able to do by the end of their 
high school years. Some students 
who take accelerated mathematics 
courses in seventh or eighth grade 
may take the Pacesetter course even 
before their final year of high school. 

Modeled after the College Board's 
successful Advanced Placement (AP) 
program, Pacesetter Mathematics 
will be appropriate for students 
preparing to go on to calculus in AP 
courses or in college; for college- 
bound students interested in 
quantitative fields such as statistics, 
accounting, or science; or for 
students planning to enter the 
work force. 

"The challenge, of course, is that 
Pacesetter seeks to raise the aca- 
demic achievement levels of the 
broadest possible range of stu- 
dents," Kimmel says. "This course 
represents the task force's best con- 
ception of a fourth -year course that 
fills that need, while at the same 
time cross-referencing and making 
specific the relevant portions of the 
NCTM framework." 

Taking a cue from the strategies 
used to develop the new standards 
for mathematics, the College Board 
is collaborating with a number of 
professional subject-matter organi- 
zations to develop the Pacesetter 
courses and assessment materials, 
which will include both classroom 
assessments during the course to 
aid instruction and end-of-course 
assessments on concepts and skills. 



gj eachers: The Key to Success 

Clearly, the success of today's 
mathematics reforms rests with 
teachers. "The best assessment or 
curriculum materials in the world 
won't do any good unless we do 
something about teacher education," 
Lesh emphasizes. 

The sad fact, however, according 
to the NCTM, is that teachers are 
not yet receiving the support and 
training they need to initiate 
changes in the classroom. 

Yet the need for such training is 
acute. According to Everybody 
Counts, of the nation's 200,000 sec- 
ondary school mathematics teach- 
ers, "over half do not meet current 
professional standards for teaching 
mathematics. Probably no more 
than 10 percent of the nation's ele- 
mentary school teachers meet con- 
temporary standards for their math- 
ematics teaching responsibilities." 

One reason for the duemma is the 
inadequate supply of mathematics 
teachers, says Everybody Counts. 
"For nearly two decades, the num- 
ber of students receiving degrees in 
mathematics has been declining, 
falling roughly 50 percent from its 
peak in the early 1970s." 

Another reason, the report says, 
is "deficit financing of intellectual 
capital. When demand for mathe- 
matics in universities increased 
sharply during the last decade, most 
institutions responded either by 
increasing class size or by hiring 
underqualified temporary teachers." 

The result has been that high 
schools "have been filling classroom 
positions with teachers whose quali- 
fications are substandard." Elemer- 
tary school teachers are also inade- 
quately prepared, says the report. 
"...Typically, they take only one of 
the four courses in mathematics rec- 
ommended as appropriate prepara- 
tion for teaching elementary school 
mathematics." 

As the "new math" experiment 
proved, short-term methods will not 
solve so deep a problem. Real 
change requires long-term, long- 
range, planned, and consistent 
training for teachers. 

"The success of the standards 
depends on two things," says Jones. 
"How well they will be carried off 
by the classroom teacher, and how 
well the classroom teacher will be 
supported to be able to do it. That 
means formal training. And it 
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means monetary support for 
that training." 

Davis and his staff at Rutgers 
University now run one of the few 
teacher training programs in the 
country that attempts to provide the 
sort of in-depth, on-the-job training 
that seems to be needed, but he 
finds that his program is frequently 
misunderstood. 

"Our business here is what's 
called teacher development, and it's 
becoming clear to me that people 
don't realize what this is like. Peo- 
ple will ask for a three-day work- 
shop for teachers to train them in 
the new material, but that's not 
even in the right ball park. Our peo- 
ple have worked with some teachers 
for seven years now," Davis says. 

"I tell people that this is a lot 
more like psychoanalysis than it is 
like teiling somebody a new recipe. 
I can tell you a new recipe fast, but 
teachers have to reflect on what 



they do, and whether there's some 
other way to do it. What often 
happens now is that people concen- 
trate on the more trivial aspects of 
it. I think that people have been 
underestimating the magnitude of 
the task." 

"School districts aren't usually 
able to provide the kind of in-service 
support teachers need," Jones notes, 
"even if their leadership may 
embrace the concept. And if they 
don't come up with the program to 
support teachers and bring them 
along, then it's not as likely to be as 
successful. 

"The standards are not a cure-all 
in and of themselves — it takes a lot 
of patience and a lot of time, and a 
firm and long commitment to be 
able to realize all of this." 

Teachers should recognize, 
however, that they do not need to be 
experts in the new standards in 
order to begin using them, Cole 
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points out. The teachers will be 
learning, too, but in doing so, they 
can show their student that learn- 
ing is not just a child's activity, it's 
every person'3 activity. 

"I think if students realized the 
teachers are learning, just as they 
are trying to learn, it would set a 
different notion of what teaching 
and learning is," she says. 

When children see that the 
teachers are experimenting with 
and learning new information, 
Cole says, w It would change the 
notion of teachers telling students 
what is right and what is wrong. 
After a while, you begin to realize 
that right-wrong is net totally 
the question. 

"Looking at different aspects of a 
complex problem is one of the things 
the more complex performance exer- 
cises are able to explore. Instead of 
asking/Is it right?' we need to ask 
instead, 4 Is this a good enough solu- 
tion, or would there have been a 
better way to do this? m 

"My view is that elementary 
teachers make a big, big difference," 
says Jones. "I think just by their 
attitude about mathematics, they 
can set a tone very early that is hard 
to reverse. My hat's off to them — 
they have to teach everything. But 
it would be nice if they could give 
math a fair shake. 

"There are workshops and pro- 
grams available for elementary 
teachers, but a large percentage of 
elementary school teachers can't 
take advantage of them. Their local 



school boards don't send them to 
those meetings. There are good 
materials and lots of resources out 
there, but many classroom teachers 
simply don't have access to them." 

The Mathematical Sciences 
Education Board addressed this 
dilemma in its 1991 publication 
Counting On You, which was 
directed to school boards, adminis- 
trators, and legislators. 

"The key is teacher professional- 
ism, 1 * the report states, "the assump- 
tion by teachers of more complete 
responsibility for the quality of edu- 
cation and the simultaneous provi- 
sion by society of the respect and 
support teachers require to get the 
job done.... 

To ensure that mathematics 
education in our schools is of the 
highest caliber, we must have 
well-prepared teachers who have 
the ability and authority to change 
within reasonable bounds the 
nature of their own roles and the 
nature of their classroom 
environments.... as schools evolve 
from a model with teachers as hired 
hands to one in which teachers func- 
tion as professional educators, they 
should welcome the challenge to 
implement national standards for 
mathematics education." 
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